Exponential Integration for Hamiltonian Monte Carlo
نویسندگان
چکیده
We investigate numerical integration of ordinary differential equations (ODEs) for Hamiltonian Monte Carlo (HMC). High-quality integration is crucial for designing efficient and effective proposals for HMC. While the standard method is leapfrog (Störmer-Verlet) integration, we propose the use of an exponential integrator, which is robust to stiff ODEs with highly-oscillatory components. This oscillation is difficult to reproduce using leapfrog integration, even with carefully selected integration parameters and preconditioning. Concretely, we use a Gaussian distribution approximation to segregate stiff components of the ODE. We integrate this term analytically for stability and account for deviation from the approximation using variation of constants. We consider various ways to derive Gaussian approximations and conduct extensive empirical studies applying the proposed “exponential HMC” to several benchmarked learning problems. We compare to state-of-the-art methods for improving leapfrog HMC and demonstrate the advantages of our method in generating many effective samples with high acceptance rates in short running times.
منابع مشابه
MATHEMATICAL ENGINEERING TECHNICAL REPORTS Hamiltonian Monte Carlo with Explicit, Reversible, and Volume-preserving Adaptive Step Size Control
Hamiltonian Monte Carlo is a Markov chain Monte Carlo method that uses Hamiltonian dynamics to efficiently produce distant samples. It employs geometric numerical integration to simulate Hamiltonian dynamics, which is a key of its high performance. We present a Hamiltonian Monte Carlo method with adaptive step size control to further enhance the efficiency. We propose a new explicit, reversible...
متن کاملPhase Equilibria of the Modified Buckingham Exponential-6 Potential from Hamiltonian Scaling Grand Canonical Monte Carlo
متن کامل
Exponential Integration for Hamiltonian Monte Carlo Supplementary Material
The one-step formulation in (7) defines a discrete flow Φh : (ri, ṙi) 7→ (ri+1, ṙi+1) which is by definition timereversible if Φh ◦ Φ−h = I . Consequently, the integration (ri+1, ṙi+1) = Φh(ri, ṙi) is time-reversible iff exchanging i ↔ i + 1 and h ↔ −h leaves it unaltered. By straightforward calculation, this is fulfilled for the choice ψ(·) = sinc(·)ψ1(·) and ψ0(·) = cos(·)ψ1(·). Moreover, Φh ...
متن کاملGeometric integration of conservative polynomial ODEs ✩
We consider systems whose Hamiltonian is of the form H(q,p) = 2p + V (q), where the potential V is either cubic or quartic with no cubic terms. For most of these systems (in the measure sense) we give an explicit numerical integration method that preserves both phase space volume and the value of the Hamiltonian. This is exemplified in the Hénon–Heiles system. An application is to the hybrid Mo...
متن کاملGradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC). On target densities where classical HMC is not an option due to intractable gradients, KMC adaptively learns the target’s gradient structure by fitting an exponential family model in a Reproducing Kernel Hilbert Space. Computational costs are reduced by two novel effi...
متن کامل